Digital watermarking is a process for modifying physical or electronic media to embed a machine-readable code into the media. The media may be modified such that the embedded code is imperceptible or nearly imperceptible to the user, yet may be detected through an automated detection process. Most commonly, digital watermarking is applied to media signals such as images, audio signals, and video signals. However, it may also be applied to other types of media objects, including documents (e.g., through line, word or character shifting), software, multi-dimensional graphics models, and surface textures of objects.
Digital watermarking systems typically have two primary components: an encoder that embeds the watermark in a host media signal, and a decoder that detects and reads the embedded watermark from a signal suspected of containing a watermark (a suspect signal). The encoder embeds a watermark by altering the host media signal. The reading component analyzes a suspect signal to detect whether a watermark is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.
Several particular watermarking techniques have been developed. The reader is presumed to be familiar with the literature in this field. Particular techniques for embedding and detecting imperceptible watermarks in media signals are detailed in the assignee's U.S. Pat. Nos. 5,862,260, 6,614,914, and 6,674,876, which are hereby incorporated by reference.
One application of digital watermarking is to uniquely serialize copies of a content item (e.g., song, music track, video program, movie, etc.). One way to serialize copies of the content item is to embed unique watermarks in each copy. In some cases, this is referred to as a transaction watermark (or transactional watermarking) because the watermark is associated with a transaction of the copy (e.g., a distribution of a copy which is serialized at the sender or receiver). The watermark itself may include unique transaction information or instead may be a code unique to the transaction and associated with transaction information in a database. In some cases, this is referred to as “forensic tracking” because the unique copy can be tracked or traced back to particular point in distribution (either a user that re-distributes, or the original distributor of the copy). For the sake of clarity, we refer to this form of watermarking to generate a unique copy as content serialization. We refer to the information associated with the unique copy as transaction information, which includes any information associated with the transaction of the copy. Depending on privacy considerations, the transaction information may or may not have information uniquely associated with a particular receiver or re-distributor of the unique copy, such as a user ID, account ID, device ID, domain ID, etc.
Since transaction information includes information specific to a transaction, it includes information that is not available prior to the transaction. This poses a technical challenge because this information cannot be associated with a unique copy until the transaction information is available. If the unique copy is generated after a request is made for the content item, the process of generating it and associating it with transaction information cannot substantially increase the latency of the transaction. Latency refers to the time between a request for a content item and its delivery to a requester. For example, in the case of a digital download of content from the Internet, the process of serializing a copy must not add substantial processing time or resource consumption after the user has requested a content item and before the user's copy is available for playback on the user's device.
To alleviate the processing required after the transaction request, certain aspects of the serializing process may be performed in advance. One approach is to create different copies of the content item in advance, divide each copy into the same content blocks, and then watermark the same blocks of content from the different copies with different data values (e.g., each block in one copy is marked with a data value of binary 1, while each block in a second copy is marked with a data value of binary zero). Then, at the time of the transaction, a content multiplexer selects blocks from the pre-embedded copies to create a new copy that is perceptually the same as the master and the pre-embedded copies, but has a unique sequence of data values embedded in it.
Another approach is to pre-compute information used in creating the unique copy prior to the transaction and then complete the process of generating the unique copy at the time of the transaction using this pre-computed information to reduce processing time. Examples include processing the content item (e.g., music track, song, TV program, movie) to identify watermark embedding locations, to calculate a perceptual mask used to control the amount of modification made at the embedding locations, and/or to compute modifications or substitute values that when applied to the content at the embedding locations produce a watermarked content item.
Another approach, which may be used in combination with the above pre-processing, is to create unique copies in advance of a request (e.g., by pre-embedding unique watermarks into distinct copies of the content item). At the time of the transaction, the content distributor provides the unique copy immediately to the requester, and then associates the unique watermark in that copy with the transaction in a transaction record database. As noted, depending on the application, this transaction record may include user information of the requester to associate the copy with the user through the unique watermark embedded in the copy. The unique watermark can be a watermark with a unique message payload (such as a serial number) or a unique pattern (such as a pseudo random pattern uniquely generated for each copy). This approach can be effective but has the drawback that extra storage is required to store uniquely serialized copies of the content item. Predictive analysis and load balancing can be used as described further below to generate serialized copies based on expected and actual popularity of a content item and then queue the copies for transmission in the edge servers of a content delivery network. While such prediction and load balancing can reduce the need to keep a large “inventory” of uniquely serialized copies, it still poses the problem that when a requester submits a request for a copy that is not in the inventory (e.g., not buffered, cached or queued for distribution), the process of generating the serialized copy must occur after the request, potentially adding to the latency of the response to the request.
An additional challenge associated with generating unique copies by altering the content item is that it may create a conflict with other information previously embedded in that content item. Digital watermarks are embedded in content for a variety of applications that do not require a unique watermark in each copy. Instead, a watermark or set of watermarks can be the same in each unique copy made from the master copy. These watermarks can be, and typically are, embedded in the master copy prior to generation of copies for distribution. For clarity, we will refer to a watermark that is the same for at least a set of unique copies of a content item to be the master copy watermark. These watermarks include watermarks that carry a content identifier (e.g., an identifier that identifies the master content item, but not individual copies of the master content item), copyright information and status, usage rights, device control flags or instructions, etc. All of this metadata associated with the master copy can be embedded in advance into the master copy from which copies are made for distribution. We will call unique watermarks embedded in the unique copies of the master copy as unique copy watermarks. The amount of metadata that needs to be embedded in either master or unique copy watermarks can be reduced by embedding an index to a metadata database that stores information about the master or unique copy. This enables the metadata to be updated over time.
The transaction information can only be associated with a unique copy after individual copies are made, even if these copies are made prior to a request. Stated another way, a copy can only be made unique (e.g., serialized) during or after that copy is generated, but not before. As such, any modification of the master copy to serialize an individual copy has the potential of altering the master copy in a manner that prevents proper reading of any previous watermark embedded in it. In addition, altering the copy may also prevent accurate identification of the master copy through the use of content fingerprinting, which identifies the master copy by its robust hash. In particular, alterations made to create unique copies that are perceptually similar may still alter the robust hash of the master copy, which has been registered in a content fingerprint database for use in identifying the master copy or its derivative copies.
One way to address this conflict is to use a process of generating unique copies that is compatible and does not conflict materially with the previous processing applied to the master copy. For example, at the time of creating the unique serialized copy, the generator of the unique copy (e.g., watermark embedder, content multiplexer, etc.) can detect a previous watermark and serialize the copy in a manner that does not conflict with the previous watermark by locating a new watermark at embedding locations that do not overlap the locations of a previously embedded watermark. These locations can be in time, space, frequency or some other domain, so long as they allow the watermarks to coexist and remain detectable by their respective detection processes.
This approach requires additional processing to detect the previous watermark, which is to be avoided if possible. To avoid this detection processing, the master copy can include metadata stored with the master copy (e.g., in a header) that specifies parameters of the previous watermark, such as its locations, perceptual mask, etc. The process of generating unique copies then uses this metadata to make alterations to create uniqueness in a manner that does not conflict with previous watermark embedding or fingerprint registration.
This disclosure describes novel methods for generating unique copies of content. One method combines the functions of the master copy and unique copy watermarks. In particular, the method generates a unique copy by varying the manner in which the master copy watermark is embedded in unique copies of a content item. In one embodiment, the master copy watermark is repeated within the content item and its location is varied in a unique pattern that comprises the unique copy watermark. The unique copy is generated by producing a copy in which master copy watermarks are embedded in a unique pattern. For instance in one embodiment, the locations of the master copy watermarks in the content item are represented as a vector of delta values in which each delta value corresponds to the distance between a corresponding instance of the master copy watermark, and a neighboring instance of the master copy watermark. This vector is stored in a transaction record in association with transaction information, such as information identifying the master copy and information associated with the unique copy. Other ways of representing the unique pattern of attributes of the master copy watermark are also possible.
In other embodiments, properties of the master copy watermark other than its location are varied through the content. The unique copy watermark is conveyed in the vector of deltas in these properties of the master copy watermark. Some examples of these properties of the master copy watermark signal include phase characteristics (e.g., the phase shift of a watermark carrier signal), frequency magnitude characteristics, etc. These properties are varied in a manner that does not alter the information carried in the master copy watermark. However, this method does provide an imperceptible variation of the content that creates a unique copy, and the unique copy watermark is conveyed in this variation. The variation in location of the instances of the master copy watermark can be in different domains, such as a spatial domain, a time domain, a time-space domain, a transform domain (including frequency transform domains), a compressed domain, etc.
To optimize performance of a content distribution system, the unique copies can be generated using one or more combinations of the pre-processing techniques described above, including pre-embedding or pre-processing the master copy of the content item.
Further features will become apparent with reference to the following detailed description and accompanying drawings.